Arrangement and method for focus monitoring in a microscope with digital image generation, preferably in a confocal microscope

ABSTRACT

Arrangement and method for focus monitoring in a microscope with digital image generation, preferably in a confocal microscope.  
     The arrangement provides an additional, rotatably supported transparent optical component in the known construction of a microscope in front of a main beam splitter arranged in the beam path in the area of a parallel illumination beam path.  
     According to the method, the images required for focus monitoring are recorded with a transparent optical component inclined by angle +α or −α. The images are checked for correlation of the image contents in the direction of beam displacement by pixel-by-pixel displacement. The determined displacement Δs at optimum correlation represents a measurement of the instantaneous focusing or defocusing.  
     The suggested solution for focus monitoring is applicable with slight adaptations for all microscopes outfitted with digital image-generating methods and arrangements. In confocal microscopes, applicability to defocusing is limited in a range of about 5 to 8×λ/NA 2 .

[0001] The present invention is directed to an arrangement and a methodby which the focus adjustment in microscopes with digital imagegeneration, preferably in confocal microscopes, can be monitored andtracked.

[0002] Various solutions are known from the prior art for determiningand tracking the optimum focal plane in microscopes.

[0003] A method for determining the optimum focal plane evaluates thebeam reflected by the object. The optimum focus position is achieved atmaximum intensity of the reflected beam. For this purpose, according toWO 00/08415, a plurality of light spots with different focal planes aregenerated. The arrangement described in GB 2 321 517 for confocalmicroscopes also provides for the evaluation of the radiation reflectedby the object for detecting the optimum focus position. These solutionsare disadvantageous in that it is sometimes very time-consuming todetermine the actual optimum focal plane. Numerous measurements arerequired for this purpose because the value of the optimum focusposition can only be determined by approximation.

[0004] Further, solutions are known from the prior art in which lightfrom a spectral range that is not used for examination is used forautofocusing. An example of this is the IR autofocus system. Theessential disadvantage in solutions providing for the use of main opticsfor the IR autofocus system consists in that the main optics must beusable for a very broad spectral range which, in some cases, may extendfrom the DUV (deep ultraviolet) range to the IR range. An objectivewhich is suitable for this purpose, such as that described in DE 199 31949, can only be realized at a very high manufacturing cost.

[0005] A solution in which the autofocus system uses separate optics andnot the main optics system is described in DE 199 19 804. This solutionuses a secondary laser source for focus monitoring. The overallarrangement is substantially more extensive and complicated as a resultof the additional arrangement for focus monitoring. Further, it isdisadvantageous that the optimum focal plane must be known because theautofocus system only compares the instantaneous beam deflection to theideal beam deflection.

[0006] Also, the technical expenditure in the solution described in WO99/03011 is substantially increased by the additional optical elementsrequired for focus monitoring. In this case, focus monitoring is carriedout by the beam offset generated in defocusing by repeated reflection atthe different optical surfaces.

[0007] The technical solutions known from the prior art have thedisadvantages that the required additional light source results in amore complicated technical arrangement, that a large number ofmeasurements are required and/or that focus monitoring is usually onlypossible with reflecting objects.

[0008] Therefore, it is the object of the present invention to develop amethod and an arrangement suited to implement this method for focusmonitoring in microscopes with digital image generation, preferably inconfocal microscopes, without adding substantially to the complexity ofthe microscope construction and in which, if possible, the existingilluminating and image evaluating devices can be used to monitor thefocal plane. The focus monitoring should be applicable independent fromthe (transparent, reflecting or fluorescing) object to be examined.Moreover, it should be possible to monitor the focal plane betweennormal work processes in a very fast and simple manner without alteringthe construction.

[0009] According to the invention, this object is met by the arrangementand the method for focus monitoring in that a suitably dimensionedrotatable plane plate is arranged in front of an existing main beamsplitter in an area with parallel illumination beams, the axis ofrotation of the plane plate being situated in relation to the specimenin such a way that a displacement of the beam bundle is carried out inthe principal scanning direction or principal image direction duringrotation. The images required for focus monitoring are recorded with ahigh zoom and with a pinhole having an aperture greater than one Airy. Afirst image is recorded with a plane plate inclined by angle +α and asecond image is recorded with a plane plate inclined by angle −α. Theseimages are checked for correlation to the image contents in thedirection of beam displacement by pixel-by-pixel displacement. Thedetermined displacement Δs at optimum correlation is a measurement ofthe instantaneous focusing or defocusing. For a displacement Δs≠0,defocusing is at a distance proportional to Δs.

[0010] The suggested method for focus monitoring and the arrangementprovided for implementing this method are applicable with slightadaptations for all microscopes outfitted with digital image-generatingmethods and arrangements. In confocal microscopes, applicability todefocusing is limited in a depth range of about 5 to 8×λ/NA².

[0011] The invention will be described in the following with referenceto two embodiment examples.

[0012]FIG. 1 is a schematic view showing the construction of a confocallaser scanning microscope with the arrangement according to theinvention;

[0013]FIG. 2 is a schematic view of the method steps according to theinvention;

[0014]FIG. 3 shows the image contents of the images to be compared, witha displacement by −3 pixels;

[0015]FIG. 4 shows the image contents of the images to be compared, witha displacement by 0 pixels;

[0016]FIG. 5 shows the image contents of the images to be compared, witha displacement of +3 pixels, with the curve of the error sum; and

[0017]FIG. 6 is a schematic view showing the construction of amicroscope with a CCD matrix and the arrangement according to theinvention.

[0018] In the arrangement for focus monitoring in a confocal microscopeshown in FIG. 1, the illumination light 1 proceeding from the laserserving as illumination source reaches the specimen 12 via a collimator2, a main beam splitter 6 which is constructed as a dichroic splitter, ascanning unit 7, a scanning objective 8, a tube lens 10, and anobjective 11. The beam reflected by the specimen 12 reaches the detector15 via the objective 11, the tube lens 10, the scanning objective 8, thescanning unit 7, the main beam splitter 6, the pinhole objective 13 andthe pinhole 14. In addition to this arrangement of a known confocalmicroscope, a transparent optical component 4 with parallel beamentrance and beam exit surfaces is arranged in the area of the parallelillumination beam path 3 in front of the main beam splitter 6 in such away that the mean perpendicular of the beam entrance surface liesparallel to the centroid of the illumination light 1. This position willbe referred to in the following as the basic position. The transparentoptical component 4 is, e.g., a plane plate or a cube with an edgelength of 10 mm and is rotatable by means of a centrally controlledmotor, preferably a stepping motor. During rotation, the meanperpendicular of the beam entrance surface encloses an angle α with thecentroid of the illumination light 1. The axis of rotation 5 of thetransparent optical component 4 is preferably situated in relation tothe specimen 12 in such a way (in direction of the y-axis) that adisplacement of the beam bundle is carried out in the main scanningdirection (x-axis) of the scanning unit 7 when rotated by angle α. Thebeam bundle is accordingly displaced from position 9 a to position 9 b.The control of the transparent optical component 4, the processing ofthe recorded images, the acquisition of all output data, and thecalculation and evaluation of the results are carried out by a centralcontrolling and evaluating unit (not shown).

[0019] In the method according to the invention for focus monitoring ina confocal laser scanning microscope, the image section contains therelevant object data. The individual method steps are described withreference to FIG. 2. An image section with about 16 rows and 512 columnsis sufficient for the method (see a) in FIG. 2). The images required forfocus monitoring are recorded at a high zoom and a pinhole 14 which isopened to approximately 4 to 5 Airy. A high zoom means that a smallerobject area is imaged in the selected image section of 512×16 pixels. Ina laser scanning microscope, this is achieved by decreasing the scanningangle of the scanning unit 7, namely, until the image length in theintermediate image plane is about 1 mm. During normal image processing,the transparent optical component 4 remains in its basic position and isnot rotated, i.e., the mean perpendicular of the beam entrance surfacelies parallel to the centroid of the illumination light 1. Thetransparent optical component 4 is rotated for monitoring focus suchthat the mean perpendicular of the beam entrance surface encloses anangle α with the centroid of the illumination light 1.

[0020] A first image is recorded with the transparent optical component4 rotated by angle α₁≈+20° and a second image is recorded with thetransparent optical component 4 rotated by angle α₂≈−20°. A third imageis recorded, in addition, also with the transparent optical component 4rotated by angle α₁≈+20°, for suppression of drift influences. Acondition for the images to be recorded is: α₁=−α₂.

[0021] The average column value determined from the first and thirdimages is checked by the central controlling and evaluating unit forcorrelation of image contents in the direction of the beam displacementby pixel-by-pixel displacement.

[0022] In step b) of FIG. 2, the quantity of data of the two imagesections to be compared is reduced (from 512×16 to 512×1) by averagingthe columns. In the next step c), the differences in illuminationbetween the right-hand and left-hand image border are compensatedmathematically by a shading correction using a lowpass of approximately10-15 pixels. This is carried out by calculating and compensating therise in the intensity functions of the two images. Depending on whetherthe maximum is at a_(n) or at b_(n), the formula for determining therise changes:$A = {{\frac{b_{n} - a_{n}}{bn}\quad {or}\quad A} = \frac{a_{n} - b_{n}}{a_{n}}}$

[0023] The calculation is carried out for the two images to be compared.As a result, the image contents have the same rise and their differencesin illumination are accordingly compensated. The minimum and maximumbrightness are determined in these corrected images, and the images areaccordingly scaled. For this purpose, the values of all pixels of everyimage are summed (Σ₁ and Σ₂) according to step d). Next, every pixel ofthe first image is multiplied by the factor (256*512/Σ₁) and every pixelof the second image is multiplied by the factor (256*512/Σ₂). The number256 is the scaling factor and 512 is the number of pixels in an image.The existing differences in brightness are compensated by means of thismethod step. The contents of these images which are corrected forillumination and scaled are compared by pixel-by-pixel displacement incolumn direction and the differences are summed quadratically for eachdisplacement until the displacement at which the sum of the differencesis a minimum is found. In step e) according to FIG. 2, this minimum isfound at a displacement of about 0 pixels. The location of the bestmatch to subpixels can be interpolated exactly from the curve of theerror sum. The pixel-by-pixel subtraction of the images to be comparedshould be carried out for displacements Δs in a range of up to a maximum10 pixels in each direction.

[0024] The magnitude of the displacement Δs in a given deviceconstruction depends upon the objective used, upon the specimen(reflecting or transparent) and to a small extent upon the wavelengthand adjusted state of the device.

[0025] All factors are determined by design with the exception of theadjustment state and can be determined in high-quality microscopes byinterrogating the adjusted configuration or can be determined bycalibration. The resulting value determined for the displacement Δs ofthe two images is accordingly proportional to the amount of thedeviation Δz from the optimum focal plane and can be calculated.

[0026] Focusing is optimum with a displacement of the two images of Δs=0as is shown in the example according to FIG. 4. In contrast, with adisplacement of the two images of Δs≠0, there is a defocusing by adistance proportional to Δs. FIG. 3 and FIG. 5 show displacements Δs by+3 pixels and −3 pixels by way of example.

[0027] In order to accelerate the process, it is advantageous to averagethe columns before compensating the differences in illumination andbrightness between the image contents of the images to be compared.Therefore, only image sections of 512×1 pixels need be processed in thefurther method steps. This does not impair the accuracy of the methodfor monitoring focus in a microscope.

[0028] Further, the method according to the invention for monitoringfocus is also applicable when rotation of the transparent opticalcomponent 4 results in a displacement of the beam bundle from position 9a to position 9 b at an angle β relative to the main scanning or mainimage direction. In this connection, the checking of the correlation ofthe image contents is carried out in the direction of beam displacementby means of pixel-by-pixel displacement, but separately in direction ofthe main scanning direction or main image direction (x-axis) and indirection of the image plane (y-axis) vertical thereto. The compensationof the differences in illumination and brightness and the pixel-by-pixelsubtraction for each displacement Δs carried out in pixel steps islikewise carried out separately in direction of the x-axis and y-axis.The determination of the resulting displacement Δs_(Res) in direction ofangle β is determined by means of the results of the displacement Δsdetermined in both directions using the Pythagorean theorem.

[0029] A value for the deviation Δz from the optimum focal plane isdetermined from the displacement Δs that can be interpolated exactlyfrom the error sum to subpixels.

[0030] In the arrangement shown in FIG. 6 for focus monitoring in amicroscope with CCD matrix 16, the illumination light 1 reaches thespecimen 12 via a lens 18, an aperture diaphragm 19, a field diaphragm17, a collimator 2, a beam splitter 6 which is partially transparentwith respect to intensity or spectrum, and an objective 11. The beamreflected by the specimen 12 reaches the CCD matrix 16 via the objective11, the beam splitter 6, the tube lens 10 and preferably by a zoomsystem 20. A light source or laser source can be used as illuminationsource. In addition to this known arrangement, a transparent opticalcomponent 4 is arranged in the area of the parallel illumination beampath 3 in front of the beam splitter 6 in such a way that the meanperpendicular of the beam entrance surface lies parallel to the centroidof the illumination light 1. The transparent optical component 4 has theshape of a square or a cylinder with a thickness from 10 to 25 mm in thedirection of the illumination light 1 and is rotatable by means of acentrally controlled motor, preferably a stepping motor. Duringrotation, the mean perpendicular of the beam entrance surface enclosesan angle α with the centroid of the illumination light 1. The axis ofrotation 5 of the transparent optical component 4 is preferably situatedin relation to the specimen 12 in such a way that a displacement of thebeam bundle (9 a to 9 b) is carried out in the main scanning imagedirection. The axis of rotation 5 is accordingly oriented to the CCDmatrix 16 in y-direction. The beam bundle is accordingly displaced fromposition 9 a to position 9 b. The control of the transparent opticalcomponent 4, the processing of the recorded images, the acquisition ofall output data, and the calculation and evaluation of the results arecarried out by a central controlling and evaluating unit (not shown).

[0031] In the method for focus monitoring, the image section of about 16rows and 512 column contains the relevant object data. The requiredimages are recorded at a high zoom. A high zoom means that a smallerobject area is imaged in the selected image section of 512×16 pixels. Ina microscope with a CCD matrix, this is realized by an optical zoom. Thezoom system 20 can be dispensed with when a sufficiently large focallength of the tube lens 10 is selected.

[0032] The transparent optical component 4 is not rotated during normalimage processing, i.e., the mean perpendicular of the beam entrancesurface lies parallel to the centroid of the illumination light 1. Formonitoring focus, the transparent optical component 4 is rotated so thatthe mean perpendicular of the beam entrance surface encloses an angle awith the centroid of the illumination light 1.

[0033] A first image is recorded with the transparent optical component4 rotated by angle α₁≈+20° and a second image is recorded with thetransparent optical component 4 rotated by angle α₂≈−20°. A third imageis recorded in addition, also with the transparent optical component 4rotated by angle α₁≈+20°, for suppression of drift influences. Acondition for the images to be recorded is: α₁=−α₂.

[0034] The mean column value determined from the first image and thirdimage is checked by the central controlling and evaluating unit forcorrelation of the image contents with the second image in the directionof the beam displacement by pixel-by-pixel displacement. The valuedetermined corresponding to the description in FIG. 1 for thedisplacement Δs of the two images is accordingly proportional to theamount of the deviation Δz from the optimum focal plane and can likewisebe calculated.

[0035] By means of the arrangement according to the invention, focusmonitoring can be implemented very quickly and simply also between thenormal work processes. A change in the illumination device or evaluatingdevice is not required for this purpose. The monitoring of a refocusingcarried out beforehand is possible at any time without a greatexpenditure of time by evaluating preferably three images.

[0036] The sensitivity of the method depends on the orientation of thestructures of the specimen to be examined and is maximum in structuresextending perpendicular to the main scanning or main image direction,very good in punctiform structures and minimal in structures extendingin the main scanning or main image direction.

1. Arrangement for focus monitoring in a microscope with digital imagegeneration, preferably in a confocal microscope, in which a parallelillumination beam path is provided in front of a main beam splitterarranged in the beam path, characterized in that a transparent opticalcomponent (4) with parallel beam entrance and beam exit surfaces isarranged in the area of this parallel illumination beam path (3) infront of the main beam splitter (6) in such a way that the meanperpendicular of the beam entrance surface lies parallel to the centroidof the illumination light (1), in that this transparent opticalcomponent (4) is supported so as to be rotatable and the meanperpendicular of the beam entrance surface encloses an angle α with thecentroid of the illumination light (1) during rotation, in that the axisof rotation (5) of this transparent optical component (4) is preferablysituated in relation to the specimen (12) in such a way that adisplacement of the beam bundle (9 a, 9 b) is carried out in the mainscanning direction or main image direction during rotation, and in thatthere is a central controlling and evaluating unit for controlling thetransparent optical component (4), for processing the recorded images,for acquisition of all output data, and for calculating and evaluatingthe results.
 2. Arrangement for focus monitoring in a microscopeaccording to claim 1, characterized in that the transparent opticalcomponent (4) has the shape of a plane plate, a cube, a square or acylinder, and in that the transparent optical component (4) has acentrally controlled motor, preferably a stepping motor.
 3. Arrangementfor focus monitoring in a microscope according to claims 1 and 2,characterized in that a laser scanning device is arranged in the mainbeam path behind the main beam splitter (6) for digital imagegeneration.
 4. Arrangement for focus monitoring in a microscopeaccording to claims 1 to 3, characterized in that a CCD matrix (16) isarranged in the beam path behind the zoom system (20) as a receiver fordigital image generation.
 5. Method for focus monitoring with digitalimage generation, preferably in a confocal microscope, particularly foroperation of an arrangement, according to at least one of the precedingclaims, in which the image section contains the relevant object data,characterized in that the images required for the focus monitoring arerecorded with a high zoom and with a pinhole (14) having an aperturegreater than one Airy, in that a first image of the relevant object datais recorded with a transparent optical component (4) that is rotated byangle +α, in that a second image of the relevant object data is recordedwith a transparent optical component (4) that is rotated by angle −α, inthat the two images are compensated with respect to their differences inillumination and brightness, in that the two images are checked forcorrelation of image contents in the direction of the beam displacementby pixel-by-pixel displacement, in that the displacement Δs at which thecorrelation has a maximum is exactly interpolated as the location of thebest match to subpixels, and a value for the deviation Δz from theoptimum focal plane is determined from this displacement Δs while takinginto account the given output values.
 6. Method for focus monitoring ina microscope according to claim 5, characterized in that a third imageof the relevant object data is recorded with a transparent opticalcomponent (4) rotated by angle +α, in that the average column value isdetermined from the third image and the first image, and in that theaverage value of the first and third images with the second image ischecked for correlation of image contents in the direction of the beamdisplacement by pixel-by-pixel displacement.
 7. Method for focusmonitoring in a microscope according to claims 5 and 6, characterized inthat a zoom of approximately 5.6 to 8 is adjusted, so that the imagelength in the intermediate image plane is approximately 1 mm, and inthat the pinhole (14) is opened to approximately 4 to 5 Airy.
 8. Methodfor focus monitoring in a microscope according to claims 5 to 7,characterized in that the angle a has a value of approximately 20°, sothat the centroid of the illumination light (1) in the objective pupilcan be displaced by approximately half of the radius of the objectivepupil, and in that the condition α₁=−α₂ applies.
 9. Method for focusmonitoring in a microscope according to claims 5 and 8, characterized inthat the compensation of the differences in illumination and brightnessand the correlation of the image contents of the images are carried outin that the differences in illumination between the right-hand imageborder and the left-hand image border in both images are mathematicallycompensated by averaging an area of about 10 to 15 pixels, in that theminimum and maximum brightness is determined in the two corrected imagesand the images are scaled, in that the contents of these images whichare corrected for illumination and scaled are subtracted one from theother pixel by pixel for each displacement Δs carried out in pixelsteps, and in that the sum of the squares of the respective differencevalues is determined for every displacement Δs as an error sum. 10.Method for focus monitoring in a microscope according to claims 5 to 9,characterized in that the columns are averaged before compensating forthe differences in illumination between the image contents.
 11. Methodfor focus monitoring in a microscope according to claims 5 to 10,characterized in that the pixel-by-pixel subtraction of the contents ofthe images which are corrected for illumination and scaled is carriedout for displacements Δs in an area of up to 10 pixels in everydirection.
 12. Method for focus monitoring in a microscope according toclaims 5 to 11, characterized in that the location of the minimum isexactly interpolated from the curve of the error sum which correspondsto a parabola in a very close approximation in the vicinity of theminimum.
 13. Method for focus monitoring in a microscope according toclaims 5 to 12, characterized in that for images in which the rotationof the transparent optical component (4) results in a displacement ofthe beam bundle (from 9 a to 9 b) at an angle β relative to the mainscanning direction or main image direction, the checking of thecorrelation of the image contents is carried out in the direction ofbeam displacement also by means of pixel-by-pixel displacement, butseparately in direction of the main scanning direction or main imagedirection and in direction of the image plane vertical thereto, in thatthe compensation of the differences in illumination and brightness andthe pixel-by-pixel subtraction for each displacement Δs carried out inpixel steps is carried out separately in direction of the main scanningdirection or main image direction and in direction of the image planevertical thereto, in that the displacement Δs_(Res) in direction ofangle β is determined by means of the results of the determination ofthe displacement Δs carried out in both directions using the Pythagoreantheorem, in that the displacement Δs in which the error sum has aminimum is exactly interpolated as the location of the best match tosubpixels, and in that a value for the deviation Δz from the optimumfocal plane is determined from this displacement Δs while taking intoaccount the given output values.